Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-7897] Improbe type for jdbc/"unsigned bigint" #6789

Closed

Conversation

rtreffer
Copy link
Contributor

The original fix uses DecimalType.Unlimited, which is harder to
handle afterwards. There is no scale and most data should fit into
a long, thus DecimalType(20,0) should be better.

@rtreffer
Copy link
Contributor Author

@viirya pushed the original patch, any objections?

@viirya
Copy link
Member

viirya commented Jun 13, 2015

@rtreffer Yes, looks like DecimalType(20,0) should be enough because unsigned bigint range is 0 to 18446744073709551615.

@rxin
Copy link
Contributor

rxin commented Jun 13, 2015

Jenkins, test this please.

The original fix uses DecimalType.Unlimited, which is harder to
handle afterwards. There is no scale and most data should fit into
a long, thus DecimalType(20,0) should be better.
@rtreffer rtreffer force-pushed the spark-7897-unsigned-bigint-as-decimal branch from e99052c to 2006613 Compare June 13, 2015 09:10
@rtreffer
Copy link
Contributor Author

Oh, I'm really lucky with with jenkins, seems like it got a network error (or github quote or alike)
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/34820/console

Rebased on master to trigger another build.

@rxin
Copy link
Contributor

rxin commented Jun 13, 2015

Jenkins, ok to test.

@SparkQA
Copy link

SparkQA commented Jun 13, 2015

Test build #34828 has finished for PR 6789 at commit 2006613.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor

rxin commented Jun 13, 2015

Thanks. Merging this in.

@asfgit asfgit closed this in d986fb9 Jun 13, 2015
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
The original fix uses DecimalType.Unlimited, which is harder to
handle afterwards. There is no scale and most data should fit into
a long, thus DecimalType(20,0) should be better.

Author: Rene Treffer <treffer@measite.de>

Closes apache#6789 from rtreffer/spark-7897-unsigned-bigint-as-decimal and squashes the following commits:

2006613 [Rene Treffer] Fix type for "unsigned bigint" jdbc loading.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants